Companies are increasingly storing large volumes of unstructured natural language data. For example, an e-commerce company may maintain large electronic catalogues of items, and a wealth of unstructured data associated these items in the form of user reviews, product descriptions, etc. As another example, websites may maintain personal profiles of its users, which may include natural language data about the user.
In many cases, it may be desirable to programmatically perform semantic analysis or extract structured information from such unstructured data. For example, it may be desirable to machine analyze an item's description to determine the item's features. However, current methods and models to perform such analysis are not well adapted to capture long-range annotation dependencies in the text. Thus, current text analysis techniques do not work well when they are applied to infer attributes or structure from lengthy text. Relatedly, many current machine learning models for text analysis are too complex for their given task, a condition which results in an overfitting of the model to the training data set. The problem of overfitting is a poorly understood mechanism, and the task of tuning a model to the appropriate level of complexity to avoid overfitting remains a practical challenge. Better models for extracting structure from long text and better ways of tuning such models are generally needed.
While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include,” “including,” and “includes” mean including, but not limited to.
It will also be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the scope of the present invention. The first contact and the second contact are both contacts, but they are not the same contact.